Self calibration of extrinsic camera parameters for a vehicle camera

ABSTRACT

A system and method for calibrating a camera on a vehicle as the vehicle is being driven. The method includes identifying at least two feature points in at least two camera images from a vehicle that has moved between taking the images. The method then determines a camera translation direction between two camera positions. Following this, the method determines a ground plane in camera coordinates based on the corresponding feature points from the images and the camera translation direction. The method then determines a height of the camera above the ground and a rotation of the camera in vehicle coordinates.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates generally to a system and method for calibratinga camera on a vehicle and, more particularly, to a system and method forautomatically calibrating a camera on a vehicle that tracks points onthe ground.

2. Discussion of the Related Art

Modern vehicles sometimes include one or more cameras that provideback-up assistance, take images of the vehicle driver to determiningdriver drowsiness or attentiveness, provide images of the road as thevehicle is traveling for collision avoidance purposes, provide structurerecognition, such as roadway signs, etc. For those applications wheregraphics are overlaid on the camera images, it is critical to accuratelycalibrate the position and orientation of the camera with respect to thevehicle. Because of manufacturing tolerances, a separate end-of-linecamera calibration, or aftermarket camera adjustment, must be performedon each vehicle for such things as accurate overlay of predicted vehiclepath lines.

Some known camera systems do not provide camera calibration, but revertto a default value that may provide a couple of degrees of error. Othercamera systems provide a pre-calibration approach where points on thecamera image are hand-labeled and feature point locations are handmeasured in the vehicle coordinates, such as by providing a checkerboard pattern of the image. However, these calibration techniques aretypically time consuming and must be performed at a service location.Therefore, if the vehicle is traveling and hits a bump or some otherobstacle in the road, the camera position could be altered, where thecalibration would not be accurate until the vehicle was taken to theservice location to be corrected.

Camera calibration involves determining a set of parameters that relatecamera image coordinates to vehicle coordinates and vice versa. Somecamera parameters, such as camera focal length, optical center, etc.,are stable, while other parameters, such as camera orientation andposition, are not. For example, the height of the camera depends on theload of the vehicle, which will change from time to time. This changecan cause overlaid graphics of vehicle trajectory on the camera image tobe inaccurate. Therefore, what is needed is a camera calibration processthat automatically calibrates less stable camera parameters as thevehicle is being driven where the vehicle-camera system continuallyadapts itself over time.

SUMMARY OF THE INVENTION

In accordance with the teachings of the present invention, a system andmethod are disclosed for calibrating a camera on a vehicle as thevehicle is being driven. The method includes identifying at least twofeature points in at least two camera images from a vehicle that hasmoved between taking the images. The method then determines a cameratranslation between two camera positions. Following this, the methoddetermines a ground plane in camera coordinates based on correspondingfeatures from the images and the camera translation direction. Themethod then determines a height of the camera above the ground and arotation of the camera in vehicle coordinates.

Additional features of the present invention will become apparent fromthe following description and appended claims, taken in conjunction withthe accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a camera image of the ground adjacent to a vehicle thatincludes overlay graphics;

FIGS. 2( a) and 2(b) show illustrations of a camera image for twoconsecutive camera frames;

FIG. 3 is an illustration of a vehicle including a camera moving fromone location to another location where points on the ground are used tocalibrate the camera;

FIG. 4 is an illustration of a camera relative to the ground todetermine the ground plane;

FIG. 5 is an illustration of a camera relative to the ground todetermine the camera height above the ground;

FIG. 6 is an illustration of a vehicle relative to the ground todetermine vehicle coordinates;

FIG. 7 is an illustration of a camera relative to the ground todetermine camera rotation in vehicle coordinates;

FIG. 8 is an illustration of a camera relative to the ground todetermine calibration of the camera using two points;

FIG. 9 is an illustration of cameras at different locations relative tothe ground to determine the ground plane without vehicle traveldistance;

FIG. 10 is an illustration for defining the ground plane with multiplecamera snapshots showing three-dimensional points in the cameracoordinates <O₁> and <O₂>;

FIG. 11 is an illustration for defining the ground plane with multiplecamera snapshots showing three-dimensional points in the cameracoordinates <O₁>;

FIG. 12 is an illustration showing ground plane orientationidentification with three-dimensional points in the camera coordinates;

FIG. 13 is an illustration of ground plane orientation identificationshowing three-dimensional points in camera coordinates;

FIG. 14 is an illustration showing ground plane orientationidentification and merged ground planes;

FIG. 15 is an illustration of cameras relative to the ground todetermine camera calibration when the vehicle is turning;

FIG. 16 is an illustration of cameras relative to the ground fordetermining the ground plane;

FIG. 17 is an illustration of cameras relative to two pseudo-groundplanes for determining ground plane location; and

FIG. 18 is a flow chart diagram showing an auto-calibration process forcalibrating a camera that provides progressive iterations.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The following discussion of the embodiments of the invention directed toa system and method for calibrating a camera on a vehicle as the vehicleis traveling is merely exemplary in nature, and is in no way intended tolimit the invention or its applications or uses. For example, thepresent invention calibrates a camera on a vehicle. However, as will beappreciated by those skilled in the art, the camera calibration methodof the invention may have application for calibrating other cameras thatmay not be on a vehicle.

As discussed above, for certain camera applications, the camera imagesgenerated by the camera are overlaid with graphics, such as a vehiclepath or trajectory, to help the vehicle driver navigate the vehicle.FIG. 1 is a depiction of a camera image 10 of the ground behind avehicle (not shown) where the vehicle is backing into a parking space12. Overlay graphic lines 14 and 16 are shown on the camera image thatwould be displayed to the vehicle driver on a screen in the dashboard toshow the driver the path that the vehicle needs to take to back into theparking space 12 properly.

As will be discussed in detail below, various techniques will bedescribed for calibrating a camera depending on what information isavailable to determine the camera's height above the ground and thecamera's orientation in vehicle coordinates. In a first case, thevehicle is traveling in a straight line, stationary points on the groundare tracked in successive video frames, and an odometer or vehiclevelocity reading is provided to calculate the distance traveled by thevehicle between camera images. As will be discussed, for the first case,the algorithm determines the translation and rotation between cameracoordinates for different positions of the camera, which are measured inthe camera coordinates at the first position. The algorithm alsodetermines the ground plane representation in the camera coordinatesbased on at least two points on the ground and the camera translationdirection. The algorithm then identifies the camera's height above theground and the camera rotation in the vehicle coordinates.

FIG. 2( a) illustrates a camera image 20 of a road 22 in front of avehicle 24, where a feature point 26 is identified in the image 20. FIG.2( b) is a next frame of a camera image 28 of the road 22, where thefeature point 26 is now closer to the vehicle 24.

FIG. 3 is an illustration of a vehicle 30 traveling on the ground 34 andincluding a camera 32 taking images of the ground 34 as the vehicletravels. In this representation, the vehicle 30 travels from a firstlocation to a second location a distance t₁ from each other. The centerof the camera 32 at the first location is identified as point O₁ and thecenter of the camera 32 at the second location is identified as pointO₂. Also illustrated are ground feature points P and Q and theprojection positions p₁ and p₂ of the feature point P in a normalizedimage plane of the moving camera 32 identified by line 36.

While the vehicle 24 travels on the flat ground 34 in a straight line,the feature points P and Q are identified on the ground using, forexample, a Harris corner detector around its driving path region in thecamera image. As a result of this, there are the corresponding featurepoints P and Q in the next image frame that can be identified using apyramid Lucas-Kanade optical flow estimation algorithm. For each trackedpoint (u, v) in the image, the feature points (e.g. P) are representedas p=(x_(n), y_(n), 1) in each normalized image plane based on the givenintrinsic camera parameters. Given two adjacent frames, the trackedcorresponding feature points are represented as point pairs (e.g.<p₁,p₂>) in their corresponding normalized image planes 36. For eachtracked point, such as P, the three points, namely the camera center O₁at the first camera snapshot, the camera center O₂ at the second camerasnapshot and the ground point P define a plane in three-dimensionalspace. Because p₁ is on the light-ray O₁P, and p₂ is on the light-rayO₂P, the five points O₁, O₂, p₁, p₂ and P are co-planar. Therefore:O ₁ p ₁ ·[O ₁ O ₂ ×O ₂ p ₂]=0  (1)

There is no rotation between the positions of the camera 32 when thevehicle 30 moves in a straight line. In the camera coordinates <O₁>, therays O₁p₁ and O₂p₂ are represented by the vectors p₁ and p₂. The lineO₁O₂ is represented by the point pair vehicle translation directiont=[t₁, t₂, t₃]^(T). Using this notation, equation (1) can be rewrittenas:p ₁ ·[t×p ₂]=0  (2)

The only unknown of equation (2) is the translation direction t.Equation (2) can be rewritten in matrix format for each correspondingfeature point pair <p₁,p₂> as:

$\begin{matrix}{{{{{p_{1}^{T}\lbrack t\rbrack}_{x}p_{2}} = 0},{{{where}\lbrack t\rbrack}_{x} = \begin{bmatrix}0 & {- t_{3}} & t_{2} \\t_{3} & 0 & {- t_{1}} \\{- t_{2}} & t_{1} & 0\end{bmatrix}},{p_{1} = \begin{bmatrix}p_{1} \\p_{2} \\1\end{bmatrix}}}{and}{p_{2} = \begin{bmatrix}p_{1}^{\prime} \\p_{2}^{\prime} \\1\end{bmatrix}}} & (3)\end{matrix}$

Equation (3) is a linear equation with respect to the unknown distancest₁, t₂ and t₃ where (p′₂−p₂)t₁+(p₁−p₂′)t₂+(p₂p₁′−p₁p₂)t₃=0. Thetranslation direction t can be determined from at least two point pairs<p₁, p₂>. Because the distance between the snapshots |t₁| is given asthe vehicle travel direction, the translation direction t₁ between thetwo snapshots is identified as t₁=|t₁|t.

If there are more than two pairs of corresponding camera points <p₁, p₂>and <q₁, q₂> the total least square estimation can be used with noiseelimination to robustly estimate the translation direction t. The totalleast square estimation is used to estimate the translation direction tgiven all of the corresponding feature points. Given the estimatedtranslation direction t, the outlier feature points evaluated byequation (2) are eliminated, which are outside of three times thestandard deviation of all of the feature points. A robust translationdirection t is estimated again from the remaining feature points usingthe total least square estimation.

Once the algorithm has calculated the camera translation direction t,the algorithm then determines a ground plane representation in thecamera coordinates based on the at least two points P and Q on theground 34 and the camera translation direction t. FIG. 4 is anillustration of the camera 32 at locations O₁ and O₂ relative to aground plane 40.

Because the translation direction vector t is known, thethree-dimensional location of each point P in the camera coordinates<O₁> can be calculated from the corresponding camera point pairs <p₁,p₂> based on triangulation. Assuming that the vehicle height h does notchange during the camera calibration, the camera translation direction tis parallel to the ground plane 40. Therefore, the reconstructedthree-dimensional points on the ground 34 together with the cameratranslation direction t define the ground plane 40.

If there are more than two pairs of corresponding feature points on theground 34, the mean m is calculated, and then all of the calculatedthree-dimensional points are projected to the null space of thetranslation direction t. Principal Component Analysis (PCA) can be usedto find the dominant variation direction d of the ground feature pointsin the null space of the translation direction t. The direction d is thecross-product of the direction t and the ground plane norm n. Thedominate variation direction d, the camera translation direction t andthe feature point mean m define the ground plane 40 in the cameracoordinates <O₁>.

In reality, not all of the tracked feature points P and Q will be on theground 34. The above-ground feature points contribute to errors/noises.Therefore, the feature points that have a distance to the ground plane40 greater than a threshold are eliminated. The final ground plane isestimated from the remaining feature points.

Once the algorithm determines the ground plane 40 in the cameracoordinates, the algorithm first eliminates those points that are not onthe ground and then determines the camera height h above the ground 34.FIG. 5 is a representation of the camera 32 relative to the ground plane40 showing the vectors and points for this calculation. The point Srepresents an outlier, which is above the estimated ground plane 34. Toidentify the camera height h above the ground plane 40, the norm n ofthe ground plane 40 is calculated as n=d×t. The height h of the cameraabove the ground plane 40 can be calculated as the inner product (h=<m,n>) between the feature point mean m and the ground plane norm n.

Once the algorithm calculates the height h of the camera 32, thealgorithm identifies the camera rotation in the vehicle coordinates.FIG. 6 is a representation of a vehicle 50 identifying the vehiclecoordinate frame <x,y,z> relative to the ground plane 40. The origin tobe centered between the rear wheels of the vehicle 50 on the ground 34is defined. The x-axis is parallel to the vehicle bumpers, the y-axis isthe vehicle backing direction without steering and the z-axis is thenorm n of the ground plane 40.

Because the vehicle 50 moves in a straight line, the y-axis is parallelto the camera translation direction t. The x direction is the same asthe direction d because both are orthogonal to the y-axis in the groundplane 40. The vehicle coordinates are thus defined as <d,t,n> in thecamera coordinates <O₁>.

FIG. 7 shows the camera 32 relative to the ground plane 40 that includesthe variables for calculating the camera location in the vehiclecoordinates, where α the pitch down of the camera 32 and γ is the yaw ofthe camera 32. The rotation matrix between the camera coordinates andthe vehicle coordinates is given as:R _(v→c) =[d,t,n](p _(camera,1) =R _(v→c) p _(vehicle) +t _(v→c))  (4)

The method for determining the camera orientation in the vehiclecoordinates based on two camera snapshot frames discussed above can beextended to multiple frames of data to provide a more robust process fordetermining the camera height and rotation. FIG. 8 is an illustration ofthe camera 32 traveling along the ground 34 where images are taken atthree camera locations O₁, O₂ and O₃, as shown. For point P on theground 34, a plane in three-dimensional space can be defined by thecamera locations O₁ and O₂ and the ground point P. The points p₁ and p₂are the image projection points of the ground point P at the cameralocations O₁ and O₂. Further, the camera locations O₂ and O₃ incombination with ground point Q can define a second plane inthree-dimensional space. The points q₂ and q₃ are the image projectionpoints of the ground point Q at camera locations O₂ and O₃. Thetranslational direction distance |t₁| is the distance between points O₁and O₂ and the translational direction distance |t₂| is the distancebetween points O₂ and O₃. If there are more than two snapshots, theestimation of the camera height h and rotation can be improved. In thiscase, at least one point in each pair of the neighboring camerapositions needs to be tracked.

The camera center position O₁ at the first snapshot, the camera centerposition O₂ at the second snapshot and the ground point P define a planein three-dimensional space. Because p₁ is on the light-ray O₁P and p₂ ison the light-ray O₂P, the five points O₁, O₂, p₁, p₂ and P areco-planar. Similarly, the five points O₂, O₃, q₂, q₃ and Q are alsoco-planar. Although the translation directions t₁ and t₂ between theadjacent snapshots are measured in two different camera coordinates andthey have the same direction t in both coordinates because the vehicleis traveling in a straight line. Therefore:O ₁ p ₁ ·[O ₁ O ₂ ×O ₂ p ₂]=0

p ₁ ·[t×p ₂]=0  (5)O ₂ q ₂ ·[O ₂ O ₃ ×O ₃ q ₃]=0

q ₂ ·[t×q ₃]=0  (6)

The translation direction t=[t₁, t₂, t₃]^(T) can be determined fromequation (5) and equation (6) by at least two point pairs in total inany pair of the neighboring camera positions. Because the distancebetween the adjacent snapshots |t₁| and |t₂| is given as the vehicletravel distance, the translations t₁=|t₁|t and t₂=|t₂|t. If there aremore than two pairs of corresponding points, the total least squareestimation can be used with noise elimination to estimate a robusttranslation direction t.

In this embodiment, the algorithm determines the ground planerepresentation in the camera coordinates based on at least two points onthe ground 34 and the camera translation direction t using both of thecamera coordinates. FIG. 9 is an illustration similar to FIG. 4 showingthe camera 32 at three locations to provide this calculation fordetermining the ground plane. When there are more than two camerapositions, three-dimensional point positions can be reconstructed in thesame manner as discussed above by considering only two camera positionsfor any given point. In this case, the three-dimensional position can beconstructed with respect to any one of the camera coordinates <O₁> or<O₂>. Because the translations between camera positions are known, thecoordinates can be easily translated to the reference frame.Alternatively, the reconstructed three-dimensional position Q′ withrespect to the camera coordinates <O₁> differs from thethree-dimensional point Q only by a known translation parallel to theground. To speed up the processing, the point Q′ can be used on theground 34 instead of the point Q for the ground plane determination andcontinue the solution as discussed above. FIG. 10 is an illustrationshowing three-dimensional points in the camera coordinates <O₁> and <O₂>and FIG. 11 is an illustration showing three-dimensional points in thecamera coordinates <O₁>.

The algorithm then calculates the camera height h above the ground,defines the vehicle coordinates and calculates the camera rotation inthe vehicle coordinates in the same manner as the process discussedabove with reference to FIGS. 5, 6 and 7.

According to another embodiment of the present invention for a secondcase, the camera 32 is calibrated without knowing the distance |t|traveled between the images used to provide the calibration. Forexample, vehicle speed measurements may not be provided for some reason.In this case, the camera rotation can only be estimated in the vehiclecoordinates. As above, the translation direction t is calculated betweenthe camera coordinates for different positions of the camera 32. Thesepositions are measured in camera coordinates at the first position.

With more tracked feature points, points that are not on the groundplane 40 can be eliminated. By providing more than two camera locations,the distance d can be more accurately calculated to determine thelocation of the ground plane 40.

Although the distance traveled between the image snapshots is not known,the same method discussed above can be used to estimate the translationdirection t in camera coordinates using equations (5) and (6), given thetracked points in the images.

The three-dimensional point positions are calculated in the same manneras discussed above by considering only two camera positions for anygiven point, but it is assumed that the unknown distances betweenadjacent camera positions are unit values. The translated reconstructionpoint Q′ with respect to the camera coordinates is used to estimate theground plane norm n. Due to scale ambiguity, the reconstructed point Q′is on the horizontal plane with a different depth than Q. In otherwords, the reconstructed three-dimensional point position is on apseudo-ground plane, which has a different depth and the same plane normas the ground plane, and is determined only up to a scale factor foreach of the camera origin, as shown in FIG. 12.

Unlike the discussion above for the first case, these calculationsrequire at least two feature points for each camera pair, i.e., on eachpseudo-ground plane, to estimate a robust ground norm n, as shown inFIG. 13. Once a collection of the pseudo-ground planes (e.g. ground₁ andground₂) is obtained, the pseudo-ground planes can be merged verticallyby subtracting their mean so that all of the points are on a commonpseudo-ground plane, as illustrated in FIG. 14 showing the mergedpseudo-ground plane 56. All of the points on the merged ground plane 56are projected to the null space of the translation direction t.Principal component analysis can be used to the find the dominantvariation direction d of the ground points in the null space of thetranslation direction t. Therefore, the norm n of the ground plane isn=d×t. Thus, the ground plane norm n is identified without knowing thedistance between the cameras 32 when the images are taken. The rotationmatrix between the camera coordinates and the vehicle coordinates isdetermined by equation (4).

The discussion above for calibrating the camera 32 requires the vehicleto be moving in a straight line. However, in some applications this maybe too significant of a constraint where it may be necessary tocalibrate the camera 32 when the vehicle steering angle is changing. Thethird case has to do with providing the camera calibration as thevehicle travels where the termination of the relationship between thepoints from the snapshots is taken for two different sections of thevehicle travel where it is straight. Because the vehicle is turning inthe same plane, there is merely an in-plane rotational shift from onelocation to another, which can be taken out as if the turn did notoccur.

FIG. 15 is an illustration of the camera 32 traveling along the ground34 where snapshot images are taken at four locations O₁, O₂, O₃ and O₄.In this representation, the vehicle 10 is traveling straight betweenlocations O₁ and O₂, is turning between locations O₂ and O₃, and isagain traveling straight between locations O₃ and O₄. The translationdirection t is determined from the plane O₁, p₁, O₂, p₂ and P and fromthe ground plane O₃, q₂, O₄, q₄ and Q in the same manner as discussedabove as:O ₁ p ₁ ·[O ₁ O ₂ ×O ₂ p ₂]=0

p ₁ ·[t×p ₂]=0  (7)O ₂ q ₂ ·[O ₂ O ₃ ×O ₃ q ₃]=0

q ₂ ·[t×q ₃]=0  (8)

The location of the ground plane 40 is determined in the same manner asshown in FIG. 16. Likewise, the camera height h, the vehicle coordinates<x,y,z> and the camera rotation in the vehicle coordinates <x,y,z> arealso calculated in the same manner as discussed above.

Further, if the distance traveled between the locations O₁ and O₂ and O₃and O₄ is not known and the vehicle is turning, then the determinationof the ground plane norm n using the merged ground planes 52 and 54 canbe performed in the same manner as discussed above, as shown in FIG. 17.

According to another embodiment of the invention, the auto-calibrationprocess is progressive where it is continually being updated to providerefinements, remove noise and resize the number of feature points. FIG.18 is a flow chart diagram 60 showing an updating process for theauto-calibration process. At box 62 reliable feature points are selectedbased on their cornerness values and correspondence errors in image. Atbox 64, the selected feature points pass their three directionalpositions to the additional detected features in the image at box 66 ata next iteration. The accumulated points are provided at box 68. Acorrespondence of the newly detected feature points is then provided atbox 72 from the next image at box 70. The auto-calibration process isprovided at box 74 that provides updated feature points in threedimensional space at box 76. Ground plane model checking is thenperformed at box 78 and the ground plane is updated at box 80. Newfeatures, which fit the current ground plane model, are detected at box82 and new feature points are identified at box 84.

The foregoing discussion discloses and describes merely exemplaryembodiments of the present invention. One skilled in the art willreadily recognize from such discussion and from the accompanyingdrawings and claims that various changes, modifications and variationscan be made therein without departing from the spirit and scope of theinvention as defined in the following claims.

1. A method for calibrating a camera that is moving relative to theground, said method comprising: identifying at least two feature pointsin at least two camera images; determining a camera translation androtation between camera coordinates for different positions of thecamera at a first position of the camera; defining a ground plane incamera coordinates; determining a height of the camera above the ground,wherein determining the camera height includes calculating an innerproduct between a mean of the featured points and a norm of the groundplane; and identifying a rotation of the camera in vehicle coordinates.2. The method according to claim 1 wherein determining a cameratranslation and rotation includes identifying a location point of thecamera in the at least two images relative to a normalized image plane,identifying a three-dimensional space using the camera location pointfor the two images of the camera and one of the feature points anddetermining a camera translation direction between the two images usingthe three-dimensional space.
 3. The method according to claim 2 whereinthe translation direction is calculated where the distance between thelocations where the two images are taken is known.
 4. The methodaccording to claim 2 wherein the translation direction is estimatedwhere the distance between the locations where the two images are takenis unknown.
 5. The method according to claim 2 wherein defining a groundplane includes defining a ground plane in camera coordinates based onthe at least two feature points and the camera translation direction. 6.The method according to claim 5 wherein defining the ground planeincludes using the translation direction, the two points on the groundto determine a mean for the points on the ground, a dominant variationdirection of the ground points and a ground norm direction.
 7. Themethod according to claim 1 wherein the height of the camera is used toeliminate feature points that are not on the ground.
 8. The methodaccording to claim 1 wherein determining the camera rotation in vehiclecoordinates includes determining the camera location in vehiclecoordinates using a pitch down of the camera and a yaw of the camera. 9.The method according to claim 1 wherein identifying feature pointsincludes identifying feature points when the vehicle is traveling in astraight line.
 10. The method according to claim 1 wherein providing atleast two camera images includes providing more than two camera images.11. The method according to claim 1 wherein defining the ground planeincludes merging separate ground planes for each feature point detectedon the ground.
 12. The method according to claim 1 wherein identifyingat least two feature points includes identifying two features points fortwo camera images at one time frame and two feature points for twocamera images at another time frame.
 13. The method according to claim12 wherein the camera is moving in a straight line during the one timeframe, the camera is turning between the two time frames and the camerais moving in a straight line during the another time frame.
 14. Themethod according to claim 1 wherein the camera is on a vehicle.
 15. Amethod for calibrating a camera on a vehicle that is moving relative tothe ground, said method comprising: identifying at least two featurepoints in at least two camera images; determining a camera translationand rotation between camera coordinates for different positions of thecamera at a first position of the camera, wherein determining a cameratranslation and rotation includes identifying a location point of thecamera in the at least two images relative to a normalized image plane,identifying a three-dimensional space using the camera location pointfor the two images of the camera and one of the feature points anddetermining a camera translation direction between the two images usingthe three-dimensional space; defining a ground plane in cameracoordinates based on the at least two feature points and the cameratranslation direction, wherein defining the ground plane includes usingthe translation direction, the two points on the ground to determine amean for the points on the ground, a dominant variation direction of theground points and a ground norm direction; eliminating feature pointsthat are not on the ground plane; determining a height of the cameraabove the ground by calculating an inner product between the mean of thefeatured points and the norm of the ground plane where the height of thecamera is used to eliminate feature points that are not on the ground;and identifying a rotation of the camera in vehicle coordinates using apitch down of the camera and a yaw of the camera.
 16. The methodaccording to claim 15 wherein the translation direction is calculatedwhere the distance between the locations where the two images are takenis known.
 17. The method according to claim 15 wherein the translationdirection is estimated where the distance between the locations wherethe two images are taken is unknown.
 18. The method according to claim15 wherein defining the ground plane includes merging separate groundplanes for each feature point detected on the ground.
 19. The methodaccording to claim 15 wherein identifying at least two feature pointsincludes identifying two features points for two camera images at onetime frame and two feature points for two camera images at another timeframe, wherein the camera is moving in a straight line during the onetime frame, the camera is turning between the two time frames and thecamera is moving in a straight line during the another time frame.
 20. Amethod for calibrating a camera on a vehicle that is moving relative tothe ground, said method comprising: identifying at least two featurepoints in at least two camera images; determining a camera translationand rotation between camera coordinates for different positions of thecamera at a first position of the camera, wherein determining a cameratranslation and rotation includes identifying a location point of thecamera in the at least two images relative to a normalized image plane,identifying a three-dimensional space using the camera location pointfor the two images of the camera and one of the feature points anddetermining a camera translation direction between the two images usingthe three-dimensional space; defining a ground plane in cameracoordinates based on the at least two feature points and the cameratranslation direction; determining a height of the camera above theground by calculating an inner product between a mean of the featuredpoints and a norm of the ground plane; and identifying a rotation of thecamera in vehicle coordinates.